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1.  Introduction 


1.1  Purpose 

The  goal  of  this  research  is  to  examine  if  gunners  in  a  Future  Combat  System  (FCS)  type  of 
mounted  environment  can  effectively  maintain  local  security  (i.e.,  perform  their  gunnery  tasks) 
while  managing  their  unmanned  assets  if  their  gunnery  tasks  are  assisted  by  aided  target  recog¬ 
nition  (ATR)  capabilities.  According  to  Mitchell  (2005),  which  examined  workload  for  Mounted 
Combat  System  (MCS)  crew  members  using  Improved  Performance  Research  Integration  Tool 
(IMPRINT)  modeling,  the  gunner  is  the  most  viable  option  for  controlling  robotic  assets  compared 
to  the  other  two  positions  (i.e.,  vehicle  commander  and  driver).  Mitchell  found  that  the  gunner  had 
the  fewest  instances  of  overload  and  could  assume  control  of  the  robotic  tasks.  However,  she  also 
discovered  that  there  were  instances  in  the  model  when  the  gunner  dropped  his  primary  tasks  of 
detecting  and  engaging  targets  to  perform  robotic  control  tasks,  which  could  be  catastrophic  for  the 
team  and  mission  during  a  real  operation. 

1.2  Background 

Based  on  Mitchell’s  modeling  work,  Chen  and  Joyner  (2006)  conducted  a  simulation  experiment 
that  supported  the  modeling  results  and  showed  that  when  the  robotics  operator  must  perform 
robot  targeting  and  local  security  (i.e.,  gunner’s  tasks)  at  the  same  time,  both  workload  and 
performance  degraded.  Their  results  showed  that  the  gunner’s  target  detection  performance 
degraded  significantly  when  participants  had  to  concurrently  monitor,  manage,  or  tele-operate  an 
unmanned  ground  vehicle  (UGV)  compared  with  the  baseline  condition  when  they  only  had  to 
perform  the  gunnery  task.  As  the  robotics  task  became  more  challenging,  participants’  gunnery 
performance  degraded  accordingly.  Their  gunnery  performance  was  worst  when  they  had  to  tele- 
operate  a  robot  simultaneously.  For  the  robotic  tasks,  participants’  performance  was  lowest  when 
they  controlled  a  semi-autonomous  UGV  (only  53%  of  the  targets  were  detected).  Participants’ 
perceived  workload  was  lowest  in  the  single-task  condition.  As  the  robotics  task  became  more 
difficult,  the  workload  increased  accordingly,  with  the  tele-operation  condition  being  the  highest. 

In  the  current  study,  we  examined  if  and  how  tactile  cueing,  which  delivered  simulated  ATR 
information  to  the  operator,  enhanced  the  gunner’s  performance  in  a  multi-tasking  environment 
similar  to  Chen  and  Joyner’s  (2006).  Terrence,  Brill,  and  Gilson  (2005)  compared  spatial  auditory 
and  spatial  tactile  cues  and  found  that  participants  perceived  the  tactile  cues  faster  and  more 
accurately.  In  a  recent  U.S.  Army  Research  Laboratory  (ARL)  study  by  Krausman,  Elliott,  and 
Pettitt  (2005),  tactile  cueing  was  not  found  to  be  more  effective  than  auditory  cueing  in  terms  of 
response  time,  although  it  was  more  effective  than  visual  cueing.  Additionally,  participants  rated 
tactile  cueing  as  the  most  helpful  among  the  three  types  of  alerts.  Although  tactile  stimulation 
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does  appear  to  have  the  ability  to  impact  spatial  attention  at  a  neurophysiological  level  (Kennett, 
Eimer,  Spence,  &  Driver,  2001),  spatial  attention  has  been  found  to  have  cross-modal  links  across 
visual,  auditory,  and  tactile  inputs  (Spence  &  Driver,  1997).  The  level  of  effectiveness  of  one 
spatial  information  display  relative  to  other  display  modalities  may  depend  on  the  operational 
context  of  the  experimental  procedure  (i.e.,  the  demands  of  the  tasks).  Ho,  Tan,  and  Spence  (2005) 
found  that  vibrotactile  alerts  were  powerful  directors  of  spatial  attention  in  simulated  driving 
scenarios,  with  faster  responses  even  when  reliability  levels  made  the  alerts  spatially  non-predic- 
tive.  Segond,  Weiss,  and  Sampaio  (2005)  found  that  tactile  spatial  information  could  be  used  to 
entirely  supplant  visual  information  to  navigate  a  robot  through  a  maze,  although  navigation  was 
the  only  task  performed  in  this  case. 

1.3  Current  Study 

In  this  study,  we  replicated  the  conditions  of  Chen  and  Joyner  (2006)  and  incorporated  signals 
(tactile  or  a  combination  of  tactile  and  visual)  to  help  participants  locate  potential  threats  in  the 
immediate  environment  while  they  controlled  an  unmanned  system  in  a  divided  attention 
paradigm.  The  surrogate  gunner  primary  task  was  to  determine  the  action  to  take,  based  on  a 
visual  determination  of  whether  a  potential  threat  was  hostile  or  neutral,  while  other  tasks  were 
being  conducted  (including  the  remote  targeting  task  with  the  robot  and  a  concurrent  communi¬ 
cation  task).  It  was  hypothesized  that  tactile  signals  would  improve  performance  in  the  gunnery 
and  the  robotic  control  tasks  since  it  would  alleviate  the  draw  that  target  scanning  has  on  visual 
resources.  Effects  of  individual  difference  factors  such  as  spatial  ability  and  perceived  attentional 
control  were  also  evaluated. 


2.  Method 


2.1  Participants 

A  total  of  20  students  (4  females  and  16  males)  were  recruited  from  the  University  of  Central 
Florida  (UCF)  and  participated  in  the  study.  The  ages  of  the  participants  ranged  from  18  to  38 
( mean  [M]  =  20.95,  standard  deviation  [SD]  =  4.62).  Participants  were  compensated  $8  per 
hour  and  were  given  class  credit  for  their  participation  in  the  experiment. 

2.2  Apparatus 

2.2.1  Simulators 

2.2. 1 . 1  Tactile  Control  Unit  (TCU) 

The  experiment  was  conducted  with  a  TCU  developed  by  the  ARL’s  Robotics  Collaborative 
Technology  Alliance  (RCTA)  for  the  robotic  control  tasks.  The  TCU  is  a  one-person  crew  station 
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from  which  the  operator  can  control  several  simulated  robotic  assets  that  can  perform  their  tasks 
semi-autonomously  or  be  tele-operated  (see  figure  1).  The  operator  switched  operation  modes  and 
display  modes  through  the  use  of  a  19-inch  touch-screen  display.  A  joystick  was  used  to  manipu¬ 
late  the  direction  in  which  the  unmanned  vehicles  moved  when  in  Teleop  mode.  The  UGV  simu¬ 
lated  in  our  study  is  the  experimental  unmanned  vehicle  developed  by  the  ARL.  The  simulation 
program  used  in  this  study  was  rSAF  (robotic  semi-automated  forces),  which  is  a  version  of 
OneSAF  for  robotics  simulation. 


Figure  1.  User  interface  of  RCTA  TCU. 


2.2. 1.2  Gunnery  Station 


The  gunnery  component  was  implemented  with  an  additional  screen  and  controls  to  simulate  the 
out-the-window  (OTW)  view  and  line-of-sight  (LOS)  fire  capabilities.  The  interface  consisted  of 
a  15-inch  flat  panel  monitor  and  a  joystick  (see  figure  2).  Participants  used  the  joystick  to  rotate 
the  sensors  360  degrees,  zoom  in  and  out,  switch  between  firing  modes,  and  engage  targets.  In 
each  scenario,  there  were  10  hostile  targets  and  10  neutral  targets  (i.e.,  civilians)  scattered 
throughout  the  simulated  environment. 


Figure  2.  TCU  (left)  and  gunnery  station  (gunner’s  OTW  view)  (right). 
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2.2. 1.3  ATR  Systems  -  Tactile  and  Visual  Alerts 

To  augment  target  detection  in  the  gunnery  component,  visual  and  tactile  alerts  were  used  to  cue 
the  participant  to  the  direction  of  a  target,  as  determined  by  the  ATR.  Visually,  the  targets  con¬ 
sisted  of  icons  presented  around  the  overhead  view  diagram  of  the  participant  vehicle  in  the  lower 
right  area  of  the  screen.  The  target  icon  appeared  in  one  of  eight  possible  locations  around  the 
gunner,  corresponding  to  45-degree  increments  along  a  360-degree  azimuth.  As  the  gunner  rotated 
the  view,  the  turret  portion  of  the  vehicle  diagram  moved  along  the  eight  possible  orientations  to 
allow  the  gunner  to  place  his/her  field  of  view  (FOV)  on  the  cued  target.  Tactually,  target  posi¬ 
tions  relative  to  the  gunner  were  presented  via  eight  electromechanical  transducers  known  as 
“tactors”  (see  figure  3). 


Figure  3.  Tactile  system. 

Each  tactor  delivered  a  250-Hz  sinusoidal,  salient  (approximately  20  dB  above  threshold) 
vibrotactile  stimulus  harmlessly  to  the  skin  via  the  central  plunger.  The  eight  tactors  were 
arranged  equidistantly  on  an  elasticized  belt  worn  around  the  abdomen  just  above  navel  height. 
This  configuration  was  based  upon  research  conducted  by  Cholewiak,  Brill,  and  Schwab  (2004) 
who  found  that  additional  tactors  within  this  ring  reduced  inter-tactor  distance  and  compromised 
localization  performance.  This  configuration  had  also  proved  comfortable  to  wear  for  experimen¬ 
tal  sessions,  in  the  laboratory  and  in  the  taxing  field  conditions.  The  tactile  stimulus  parameters 
were  programmed  onto  a  battery-powered  controller  board  governing  all  eight  tactors.  This  board 
was,  in  turn,  controlled  by  a  computer  running  the  simulation  and  presenting  targets  for  the  visual 
and  tactile  conditions.  The  tactile  stimulus  had  a  300-ms  duration,  which  was  determined  based 
upon  the  simulation’s  refresh  rates  for  revising  ATR  information.  To  match  the  visual  condition  as 
closely  as  possible,  a  target  that  was  directly  behind  the  gunner  (6  o’clock  position)  caused  the 
tactor  on  the  spine  to  activate.  If  the  gunner  moved  the  turret  to  the  right,  the  vibrotactile  stimulus 
appeared  to  move  along  the  right  side  of  the  body  to  the  front-most  tactor  where  the  target  was 
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now  in  the  gunner’s  FOV.  Participants  had  an  opportunity  to  become  familiar  with  both  types  of 
signals  during  the  training  session. 

2.2.2  Communication  Task 

The  communication  task  was  administered  concurrently  with  the  experimental  scenarios.  The 
questions  included  simple  military-related  reasoning  tests  and  simple  memory  tests.  The  inclusion 
of  these  cognitive  tasks  was  for  simulating  an  environment  where  the  gunner  was  communicating 
with  fellow  crew  members  in  the  vehicle.  For  the  reasoning  tests,  there  were  questions  such  as  “if 
the  enemy  is  to  our  left,  and  our  UGV  is  to  our  right,  what  direction  is  the  enemy  to  the  UGV?” 

For  the  memory  tests,  the  participants  were  asked  to  repeat  some  short  statements  or  keep  track  of 
three  radio  call  signs  (e.g.,  Bravo  83)  and  they  had  to  report  to  the  experimenter  whether  the  call 
signs  they  heard  were  one  of  those  they  were  keeping  track.  Test  questions  were  delivered  by  a 
synthetic  speech  program,  DECTalk1.  The  questions  were  pre-recorded  by  a  male  speaker  and 
presented  at  the  rate  of  one  question  every  33  seconds. 

2.2.3  Questionnaires  and  Spatial  Tests 

A  demographics  questionnaire  was  administered  at  the  beginning  of  the  training  session  (appen¬ 
dix  A).  A  questionnaire  on  Attentional  Control  (appendix  B)  (Derryberry  &  Reed,  2002)  was 
used  to  evaluate  participants’  perceived  attentional  control  (PAC).  The  Attentional  Control  survey 
consists  of  21  items  and  measures  attention  focus  and  shifting.  The  Cube  Comparison  (Cube  for 
short)  and  the  Hidden  Figures/Pattems  tests  (Hidden  Figures  for  short)  (Educational  Testing 
Service,  2007a&b)  as  well  as  the  Spatial  Orientation  Test  (Orientation  for  short)  were  used  to 
assess  participants’  spatial  ability  (SpA).  The  Cube  test  requires  participants  to  compare,  in  3 
minutes,  21  pairs  of  six-sided  cubes  and  determine  if  the  rotated  cubes  are  the  same  or  different. 

The  Hidden  Figures  test  measures  flexibility  of  closure  and  involves  identifying  specific  patterns  or 
shapes  embedded  within  distracting  information.  The  Orientation  test,  constructed  by  Dr.  Paula 
Durlach  of  the  U.S.  Army  Research  Institute,  is  modeled  after  the  cardinal  direction  test  developed 
by  Gugerty  and  his  colleagues  (Gugerty  &  Brooks,  2004)  and  is  a  computerized  test  consisting  of  a 
brief  training  segment  and  32  test  questions.  Accuracy  and  response  time  were  automatically 
captured  by  the  program. 

Participants’  perceived  workload  was  evaluated  with  the  computer-based  version  of  National 
Aeronautics  and  Space  Administration-task  load  index  (NASA-TLX)  questionnaire  (appendix  C) 
(Hart  &  Staveland,  1988).  The  NASA-TLX  is  a  self-reported  questionnaire  of  perceived  demands 
in  six  areas:  mental,  physical,  temporal,  effort  (mental  and  physical),  frustration,  and  performance. 
Participants  were  asked  to  evaluate  their  perceived  workload  level  in  these  areas  on  10-point 
scales.  They  also  assessed  the  contribution  (i.e.,  weight)  of  each  factor  to  the  perceived  workload 
by  comparing  the  15  possible  pairs  of  the  six  factors.  According  to  Noyes  and  Bruneau  (2007), 
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computer-based  NASA- TLX  tends  to  generate  higher  workload  ratings  compared  with  the 
traditional  paper-based  survey.  However,  since  the  ratings  were  used  to  compare  the  workload 
levels  across  the  experimental  conditions,  the  elevated  ratings  should  not  affect  these  comparisons. 

The  Simulator  Sickness  Questionnaire  (SSQ,  see  appendix  D)  was  used  to  evaluate  participants’ 
simulator  sickness  symptoms  (Kennedy,  Lane,  Berbaum,  &  Lilienthal,  1993).  The  SSQ  consists  of 
a  checklist  of  16  symptoms,  each  of  which  is  related  in  terms  of  degrees  of  severity  (none,  slight, 
moderate,  severe).  A  total  severity  (TS)  score  was  derived  by  a  weighted  scoring  procedure  and 
reflected  overall  discomfort  level. 

Finally,  a  usability  questionnaire  on  the  tactile/visual  ATR  displays  was  constructed  (see  appen¬ 
dix  E).  Participants  indicated  their  level  of  reliance  on  tactile  and/or  visual  cueing  for  the  gunnery 
task  when  both  types  of  alerts  were  available.  They  also  indicated  their  perceived  usability  of  the 
ATR  displays. 

2.3  Experimental  Design 

The  overall  design  of  the  study  is  a  2  x  3  repeated  measures  design.  The  independent  variables  are 
ATR  type  (Baseline-no  alerts  versus  Tactile  alerts  only  versus  Tactile  +  Visual  alerts)  and  Robo¬ 
tics  Task  type  (Auto  versus  Teleop).  There  were  six  conditions: 

•  Auto-BL  (baseline):  No  alerts  +  control  of  a  semi-autonomous  UGV 

•  Teleop-BL:  No  alerts  +  Tele-operating  a  UGV 

•  Auto-Tac:  Tactile  alerts  +  control  of  a  semi-autonomous  UGV 

•  Teleop-Tac:  Tactile  alerts  +  Tele-operating  a  UGV 

•  Auto-TacVis:  Tactile  alerts  +  Visual  alerts  +  control  of  a  semi-autonomous  UGV 

•  Teleop-TacVis :  Tactile  alerts  +  Visual  alerts  +  Tele-operating  a  UGV 

The  reliability  level  of  the  alerts  was  100%.  However,  only  hostile  targets  were  cued,  not  the 
neutral  targets.  The  participants  had  to  detect  the  neutral  targets  on  their  own. 

2.4  Procedure 

After  being  briefed  about  the  purpose  of  the  study,  the  tasks  for  the  experiment,  and  any  risks 
involved,  participants  read  and  signed  a  consent  form.  Then  they  answered  the  Attentional  Control 
survey  and  were  administered  the  spatial  ability  tests  (i.e.,  Cube,  Hidden  Figures,  and  Orientation). 
After  these  tests,  participants  received  training,  which  lasted  approximately  2  hours.  Training  was 
self-paced  and  was  delivered  by  PowerPoint2  slides  showing  the  elements  of  the  TCU,  steps  for 
completing  various  tasks,  several  mini-exercises  for  practicing  the  steps,  and  two  exercises  for 
performing  the  robotic  control  tasks  (one  for  practicing  the  tele-operation  task  and  one  for 
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practicing  the  UGV  control  tasks).  After  the  tutorial  on  TCU,  participants  were  trained  in  the 
gunnery  tasks.  The  entire  training  session  lasted  about  2.5  hours. 

The  experimental  session  took  place  on  a  different  day  but  within  a  week  of  the  training  session. 
Before  the  experimental  session  began,  participants  were  given  some  practice  trials  and  review 
materials,  if  necessary,  to  refresh  their  memories.  After  the  refresher  training,  participants 
completed  one  combined  exercise  in  which  they  performed  all  three  tasks  (i.e.,  gunnery,  robotic 
control,  and  communication  tasks)  at  the  same  time.  At  this  point,  all  tutorial  materials  and 
information  were  removed  and  the  participants  had  to  be  able  to  perform  all  these  tasks  on  their 
own.  After  this  final  exercise,  the  experimenter  determined  if  the  participant  needed  any  further 
practice  on  the  robotic  control  tasks  or  gunnery  tasks  and  provided  some  further  training  and 
exercises  if  necessary.  Participants  had  to  demonstrate  that  they  could  recall  all  the  steps  for 
performing  the  tasks  without  any  help. 

Before  the  experimental  session  began,  participants  changed  into  one  of  the  laboratory  cotton  T- 
shirts  in  order  to  standardize  how  the  tactors  were  applied  to  the  skin.  They  chose  a  size  and  then 
were  escorted  to  the  restroom  where  they  could  change  privately.  Then  the  experimenter  asked  to 
measure  the  participant  around  the  abdomen  just  above  navel  height  so  that  the  tactile  display 
could  be  custom  fitted.  After  taking  this  measurement,  the  experimenter  arranged  the  tactors  so 
that  they  were  equidistant  for  the  participant’s  abdomen.  Once  fitted  with  the  tactile  display,  the 
participant  was  seated  in  front  of  the  gunner  monitor.  A  test  pattern  confirmed  that  all  eight  tactors 
were  working  properly  and  that  the  participant  could  readily  perceive  the  stimuli.  The  experi¬ 
menter  then  explained  the  nature  of  the  ATR  system  and  the  corresponding  visual  or  tactile  cues 
that  would  be  provided. 

In  the  experimental  trials,  participants’  tasks  were  to  use  their  robotic  asset  to  locate  targets  (i.e., 
enemy  dismounted  Soldiers)  in  the  remote  environment  and  to  find  targets  in  their  immediate  (i.e., 
MCS)  environment.  The  MCS  was  simulated  as  traveling  along  a  designated  route,  which  was 
approximately  4.3  km  and  lasted  about  15  minutes.  There  were  10  hostile  and  10  neutral  targets 
(i.e.,  civilians)  along  the  route  in  each  gunnery  scenario.  Participants  were  instructed  to  engage  the 
hostile  targets  and  verbally  report  spotting  the  neutral  targets.  In  total,  there  were  six  15-minute 
scenarios,  corresponding  to  the  six  experimental  conditions,  the  order  of  which  was 
counterbalanced  across  participants. 

There  were  two  types  of  robotics  tasks:  Auto  and  Teleop.  The  Auto  control  task  required  the 
operator  to  monitor  the  video  feed  as  the  UGV  traveled  autonomously,  to  examine  still  images 
generated  from  the  RSTA  (reconnaissance,  surveillance,  and  target  acquisition)  scans,  and  detect 
targets.  The  Teleop  task  required  the  operator  to  manually  manipulate  and  drive  the  UGV  (using  a 
joystick)  along  a  predetermined  route  using  the  TCU  to  detect  targets.  For  both  the  Auto  and 
Teleop  tasks,  upon  detecting  a  target,  participants  needed  to  place  the  target  on  the  map,  label  the 
target,  and  then  send  a  spot  report.  A  list  of  robotic  tasks  for  the  Auto  and  Teleop  conditions  is 
presented  in  table  1 . 
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Table  1.  Robotic  tasks  for  the  Auto  and  Teleop  conditions. 


Condition 

Auto 

Teleop 

Tasks 

Identify  target  or  neutral  by  analyzing  RSTA  scans 

Identify  target  or  neutral 

Verbally  report  neutrals 

Verbally  report  neutrals 

Queue  target  (i.e.,  add  to  map) 

Switch  to  Map  Display 

Switch  to  Map  Display 

Add  target  to  the  map  manually 

Label  target 

Label  target 

Submit  Spot  Report 

Submit  Spot  Report 

While  the  participants  were  performing  their  gunnery  and  robotics  control  tasks,  they  simultane¬ 
ously  performed  the  communication  task  by  answering  questions  delivered  to  them  via  DECtalk. 

There  were  2-minute  breaks  between  experimental  scenarios.  Participants  assessed  their  workload 
using  the  NASA-TLX  after  they  completed  each  scenario.  At  the  conclusion  of  all  scenarios, 
participants  were  administered  the  SSQ,  used  to  evaluate  the  severity  of  their  simulator  sickness 
symptoms.  The  participants  also  completed  a  usability  questionnaire  regarding  the  tactile/visual 
cueing  systems  at  the  end  of  the  experimental  session.  The  entire  experimental  session  lasted 
about  3  hours. 

2.5  Measures 

The  dependent  measures  include  mission  performance  (i.e.,  number  of  targets  detected  in  the 
remote  environment  by  the  robotic  asset  and  number  of  hostile/neutral  targets  detected  in  the 
immediate  environment),  communication  task  performance,  and  perceived  workload.  Since 
participants’  robotics  task  performance  in  the  Auto  condition  was  limited  by  the  capabilities  of 
the  TCU  (i.e.,  accuracy  of  the  RSTA  scans),  it  was  determined  that  only  the  performance  data 
from  the  Teleop  condition  would  be  analyzed. 


3.  Results 


3.1  Target  Detection  Performance 

3.1.1  Gunnery  Tasks 

Table  2  lists  several  measures  relating  to  target  detection.  Participants  were  designated  as  high 
SpA  or  low  SpA,  based  on  their  composite  spatial  ability  test  scores  (median  split).  A  mixed 
analysis  of  variance  (ANOVA)  was  performed  to  examine  the  effects  of  the  concurrent  robotic 
control  tasks  on  the  gunnery  task  performance  (percentage  of  hostile  targets  detected),  with  the 
Robotics  Task  condition  (Auto  versus  Teleop)  and  the  ATR  condition  (Baseline  versus  Tac  versus 
TacVis)  being  the  within-subject  factors  and  SpA  (High  versus  Low)  as  the  between-subject 
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factor.  The  ANOVA  revealed  that  ATR  condition  significantly  affected  the  number  of  targets 
detected,  F(2,  36)  =  78.623,/)  <  .001.  Post  hoc  tests  (Least  Significant  Difference  or  LSD) 
showed  that  target  detection  in  the  Baseline  condition  was  significantly  lower  than  in  the  Tac  and 
TacVis  conditions.  Participants  with  higher  SpA  had  significantly  higher  gunnery  task 
performance  than  did  those  with  lower  SpA,  F(  1,  18)  =  5.659,/)  <  .05  (figure  4). 


Table  2.  Mean  proportion  of  targets  detected  (standard  deviations  are  presented  in  parentheses). 


Measures 

Auto 

Teleop 

Baseline 

Tac 

TacVis 

Baseline 

Tac 

TacVis 

Gunnery  Task 

,550a 

M)5 

J445 

,495a 

332^ 

jw 

(Hostile  only) 

(.193) 

(.143) 

(.114) 

(.196) 

(.162) 

(.092) 

Gunnery  Task 

.485a 

Iso5 

354s 

,425c 

T893 

pM 

(Neutral  only) 

(.187) 

(.128) 

(.148) 

(.241) 

(.091) 

(.126) 

Robotic  Task 

NA 

NA 

NA 

,690a 

.780a 

.786a 

(Teleop  only) 

(.290) 

(.190) 

(.227) 

Communication 

.862a 

^90® 

^99® 

.847a 

A735 

^82® 

Task 

(.104) 

(.073) 

(.110) 

(.108) 

(.083) 

(.123) 

Note:  Statistics  with  the  same  superscript  are  not  significantly  different  from  one  another 


Figure  4.  Effects  of  spatial  ability  on  gunner’s  hostile  target  detection  performance. 

Participants’  detection  of  neutral  targets  was  also  assessed.  Since  the  ATR  only  alerted  the 
participants  when  hostile  targets  were  present,  the  neutral  target  detection  could  be  used  to  indicate 
how  much  visual  attention  were  devoted  to  the  gunnery  station.  Performance  data  from  the  Tac 
and  TacVis  conditions  were  merged  to  form  the  ATR  condition  and  were  compared  with  the  Base¬ 
line  condition.  An  ANOVA  revealed  a  significant  main  effect  for  both  Robotics,  F(l,16)  =  10.407, 
p  <  .01,  and  ATR,  F(2,32)  =  15.272,/)  <  .001  (figure  5).  Post  hoc  tests  (LSD)  showed  that  neutral 
target  detection  in  Baseline  was  significantly  higher  than  in  the  Tac  and  TacVis  conditions;  TacVis 
was  also  significantly  higher  than  Tac. 
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Figure  5.  Gunner’s  neutral  target  detection  performance. 


3.1.2  Robotics  Tasks 

Since  the  participants’  task  performance  in  the  Auto  condition  was  limited  by  the  capabilities  of 
the  TCU,  it  was  determined  that  only  the  performance  data  from  the  Teleop  condition  would  be 
included  for  the  analyses.  Performance  data  from  the  Tac  and  TacVis  conditions  were  again 
merged  to  form  the  ATR  condition  and  were  compared  with  the  Baseline  condition.  The  Baseline 
condition  was  found  to  be  significantly  lower  than  the  ATR  condition,  F(l,18)  =  5.342, <  .05. 
The  performance  of  participants  with  higher  SpA  exceeded  that  of  those  with  lower  SpA  in  the 
baseline  condition,  F(  1,1 8)  =  5.851  ,p<  .05,  but  not  in  the  ATR  conditions  (figure  6).  Similarly, 
the  performance  of  participants  with  high  PAC  exceeded  that  of  those  with  lower  PAC  in  the 
baseline  condition  (although  the  difference  was  only  marginally  significant,  F(l,18)  =  4.343, 
p  =  .052)  but  not  in  the  ATR  conditions. 


Figure  6.  Effects  of  spatial  ability  on  robotics  task  performance. 
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3.2  Communication  Task  Performance 

Performance  data  from  the  Tac  and  TacVis  conditions  were  again  merged  to  form  the  ATR 
condition  and  were  compared  with  the  Baseline  condition.  The  difference  between  these  two 
conditions  was  significant,  F(  1,  19)  =  7.416,/?  <  .05,  with  the  no  ATR  condition  lower  (figure  7). 
No  significant  correlations  between  communication  performance  and  the  individual  difference 
factors  (i.e.,  SpA  and  PAC)  were  observed. 


Figure  7.  Communication  task  performance. 


3.3  Perceived  Workload 

Weighted  ratings  of  the  scales  of  the  NASA- TLX  were  used  for  this  analysis.  Participants’  self- 
assessment  of  workload  was  significantly  affected  by  the  Robotic  condition,  F(l,  18)  =  5.212, 
p  <  .05,  as  well  as  the  ATR  condition,  F( 2,32)  =  4.30,/?  <  .05  (figure  8).  The  perceived  workload 
was  higher  in  the  Teleop  condition  (M=  70.22)  and  when  the  gunnery  task  was  unassisted  by  the 
ATR  (M=  70.54).  No  significant  correlations  between  perceived  workload  and  the  individual 
difference  factors  (i.e.,  SpA  and  PAC)  were  observed. 


Figure  8.  Perceived  workload. 
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3.4  Simulator  Sickness 

Participants’  simulator  sickness  scores  (SSQ:  three  sub-scales  and  the  total  severity  score  [TSS]) 
were  calculated,  based  on  the  formulae  in  Kennedy  et  al.  (1993)  (see  appendix  F  for  the  scoring 
procedure).  The  average  TSS  was  32.69  ( SD  =  32.87),  which  was  not  significantly  different  from 
the  TSS  reported  in  Chen  and  Joyner  (2006).  Additional,  no  significant  correlations  were  found 
between  TSS  and  attentional  control,  amount  of  sleep,  or  the  spatial  tests.  There  was  a  significant 
correlation  between  simulator  sickness  and  workload  ratings  for  tactile-visual  ATR  (two-tailed 
Pearson  r  =  .500,/?  =  .030,  N  =  19;  ratings  summated  across  robotics  conditions).  A  mixed 
ANOVA  with  simulator  sickness  as  coded  into  high  and  low  levels  (using  a  median  split)  yielded 
no  significant  findings  for  gunner  performance  across  ATR  type,  enemy/neutral  detection,  and 
robotics  task  type. 

3.5  Usability  Questionnaire 

A  usability  questionnaire  captured  participant  preferences  for  presentation  of  ATR  information. 
Following  their  interaction  with  the  ATR  systems,  65%  of  participants  responded  that  they  relied 
predominantly  or  entirely  on  the  tactile  ATR  display.  Only  15%  responded  that  they  relied  pre¬ 
dominantly  or  entirely  on  the  visual  ATR  display.  ATR  preference  was  also  significantly  corre¬ 
lated  with  attentional  control  scores  (two-tailed  Pearson  r  =  .482,/?  =  .031,  N  =  20)  as  well  as  the 
composite  score  of  the  spatial  tests  (two-tailed  Pearson  r  =  .532,  p  =  .016,  N  =  20). 

A  mixed  ANOVA  on  gunner  performance  for  detecting  enemy  targets  was  conducted  with  Robotics 
task  type  as  a  within- subject  factor  and  ATR  preference  as  a  between-subject  factor.  There  was  a 
main  effect  for  ATR  preference,  F(2,17)  =  3.995, p  <  .05  (figure  9).  Post  hoc  tests  revealed  that 
those  who  indicated  a  strong  preference  for  visual  ATR  performed  significantly  worse  than  those 
who  preferred  to  have  both  or  tactile  alone.  The  performance  for  those  who  preferred  tactile  did  not 
differ  significantly  from  those  who  indicated  that  they  relied  on  both.  Caution  must  be  taken  with 
these  results  since  the  cell  sizes  for  indicated  preferences  are  not  equal. 

A  similar  ANOVA  on  gunner  performance  for  detecting  neutral  targets  did  not  yield  any  significant 
findings  with  respect  to  ATR  preference.  However,  another  mixed  ANOVA  examining  robotics 
task  performance  with  ATR  as  a  within-subject  factor  and  ATR  preference  as  a  between-subject 
factor  did  yield  a  significant  main  effect  for  ATR  preference,  F(2,17)  =  4.177,/?  <  .05.  Post  hoc 
analyses  showed  higher  performance  for  tactile  ATR  preference  over  those  who  preferred  visual 
ATR.  No  significant  differences  were  found  between  those  who  preferred  both  and  tactile  and 
vision. 
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Figure  9.  ATR  preference  and  gunnery  performance. 


4.  Discussion 


In  this  study,  we  simulated  a  generic  mounted  environment  and  conducted  an  experiment  to 
examine  the  performance  and  workload  of  the  combined  position  of  gunner  and  robotic  operator. 
More  specifically,  we  compared  the  performance  and  workload  of  the  operator  when  his  or  her 
gunnery  tasks  were  assisted  by  ATR  capabilities  (delivered  through  tactile  cueing  or  a  combination 
of  tactile  and  visual  cueing)  versus  when  the  gunnery  task  was  unassisted.  Results  showed  that  the 
gunner’s  target  detection  performance  improved  significantly  when  his/her  task  was  assisted  by 
ATR.  Consistent  with  the  findings  of  Chen  and  Joyner  (2006),  participants’  SpA  was  found  to  be 
an  accurate  predictor  of  their  gunnery  performance.  The  performance  of  participants  with  higher 
SpA  consistently  exceeded  that  of  those  with  lower  SpA  throughout  the  scenarios. 

It  was  also  found  that  gunner’s  detection  of  neutral  targets  (which  was  not  aided  by  ATR)  was 
significantly  worse  when  s/he  had  to  tele-operate  a  robotic  asset  (versus  when  the  asset  was  semi- 
autonomous)  or  when  the  gunnery  task  was  aided  by  ATR.  This  finding  suggests  that  participants 
devoted  significantly  less  visual  attention  to  the  gunnery  station  when  their  robotic  asset  required 
tele-operation  or  when  their  gunnery  task  was  assisted  by  ATR.  On  average,  participants  detected 
45%  of  the  neutral  targets  when  there  was  no  ATR;  they  only  detected  28%  when  there  was. 
Results  of  the  current  study  are  consistent  with  automation  research  that  operators  may  develop 
over-reliance  on  the  automatic  system  and  this  complacency  may  negatively  affect  their  task 
performance  (Chen  &  Joyner,  2006;  Parasuraman,  Molloy,  &  Singh,  1993;  Thomas  &  Wickens, 
2000;  Young  &  Stanton,  2007).  It  is  worth  noting  that  these  findings,  along  with  the  results  of  the 
current  study,  do  not  necessarily  suggest  that  manual  manipulation  of  sensor  devices  be  used 
instead  of  ATR  devices.  However,  the  issue  of  over-reliance  on  these  automatic  capabilities  needs 
to  be  taken  into  account  when  one  is  designing  the  user  interface  where  these  features  are  part  of 
the  components. 
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For  the  robotics  tasks,  since  the  participants’  task  performance  in  the  Auto  condition  was  limited 
by  the  capabilities  of  the  TCU  and  therefore  could  not  indicate  their  true  performance,  only  the 
performance  data  from  the  Teleop  condition  were  used  for  the  analyses.  The  results  showed  that 
participants’  tele-operation  performance  improved  significantly  when  their  gunnery  task  was 
assisted  by  ATR.  Therefore,  ATR  was  not  only  beneficial  for  the  automated  task  (i.e.,  gunnery) 
but  also  the  concurrent  task  (i.e.,  robotics).  Additionally,  it  was  evident  that  ATR  was  more 
beneficial  for  enhancing  the  concurrent  robotics  performance  for  those  with  lower  SpA  than  for 
those  with  higher  SpA.  Similarly,  ATR  appeared  to  benefit  those  with  lower  PAC  more  than  those 
with  higher  PAC.  When  ATR  was  available  to  assist  those  operators  with  low  SpA  and  PAC,  the 
performance  of  their  concurrent  task  was  improved  to  a  similar  level  as  those  with  higher  SpA  and 
PAC.  These  results  may  have  important  implications  for  system  design  and  personnel  selection  for 
the  Army’s  FCS  program. 

The  participants’  communication  task  performance  improved  when  their  gunnery  task  was  aided 
by  ATR.  Again,  this  result  suggests  that  ATR  not  only  enhanced  the  tasks  it  was  designed  for,  but 
it  also  benefited  concurrent  tasks.  It  also  shows  that  our  cognitive  communication  task  was  sensi¬ 
tive  to  the  task  load  manipulations  we  implemented  for  the  primary  task  (ATR  versus  no  ATR). 
However,  unlike  Chen  and  Joyner  (2006),  those  with  higher  PAC  were  not  found  to  outperform 
those  with  lower  PAC.  Further  research  is  needed  to  examine  the  relationship  between  attentional 
resource  management  and  concurrent  task  performance. 

The  participants’  perceived  workload  was  found  to  be  affected  by  the  type  of  concurrent  robotic 
task  as  well  as  whether  their  gunnery  task  was  aided  by  ATR.  They  had  a  higher  workload  level 
when  their  gunnery  task  was  unassisted  by  ATR.  They  also  experienced  significantly  higher 
workload  when  they  tele-operated  the  robotic  asset.  These  results  are  consistent  with  Chen  and 
Joyner  (2006)  and  Schipani  (2003),  which  evaluated  robotic  operator  workload  in  a  field  setting. 
Although  many  of  the  ground  robotic  assets  in  the  Army’s  FCS  program  will  be  semi-autonomous, 
tele-operation  will  still  be  an  important  part  of  any  missions  involving  robotic  assets  (e.g.,  when 
robots  encountered  obstacles  or  other  problems).  The  higher  workload  associated  with  tele¬ 
operation  needs  to  be  taken  into  account  when  one  is  designing  the  user  interfaces  for  the  robotic 
assets. 

Reported  simulator  sickness  did  not  serve  as  a  meaningful  covariate  with  performance  on  both  the 
gunner  task  and  tele-operating  the  UGV.  Overall,  participants’  simulator  sickness  seemed  slightly 
more  severe  than  in  Chen  and  Joyner  (2006),  although  the  difference  was  not  statistically  signifi¬ 
cant.  We  did  not  find  differences  between  those  with  higher  and  lower  attentional  control,  as 
reported  in  Chen  and  Joyner  (2006). 

The  significant  positive  correlation  of  ATR  preference  with  the  composite  score  of  spatial  tests  is 
interesting,  since  it  appeared  that  as  ATR  ratings  tended  toward  considerable  reliance  on  the  tactile 
display,  there  was  a  concurrent  shift  with  higher  performance  on  the  spatial  tests.  Perhaps  those 
with  higher  spatial  ability  can  more  easily  employ  the  spatial  tactile  signals  in  the  dual  task  setting 
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and  therefore  have  a  stronger  preference  for  something  that  makes  the  gunner’s  task  easier  to 
complete.  Individuals  with  lower  spatial  ability,  on  the  other  hand,  may  have  not  used  the  spatial 
tactile  cues  to  their  full  extent  and  therefore  continued  to  prefer  the  visual  ATR  display.  Accord¬ 
ing  to  Kozhevnikov,  Hegarty,  and  Mayer  (2002),  visualizers  with  lower  spatial  ability  tend  to  rely 
on  iconic  imagery  while  those  with  higher  spatial  ability  tend  to  prefer  using  spatial-schematic 
imagery  while  solving  problems.  Therefore,  it  is  likely  that,  in  our  study,  those  who  preferred 
visual  ATR  displays  may  have  been  more  iconic  in  their  mental  representations.  However,  this 
preference  may  have  caused  degraded  target  detection  performance  because  of  more  visual 
attention  being  devoted  to  the  visual  ATR  display  instead  of  the  simulated  environment.  In 
contrast,  those  who  were  more  spatial  relied  on  the  directional  information  of  the  tactile  display 
to  help  them  with  the  visually  demanding  tasks,  which  resulted  in  a  more  effective  performance. 
Additional  research  using  multimodal  spatial  information  in  this  experiment  test  bed  could  help 
to  determine  the  appropriate  methods  for  displaying  new  technological  information  advancements 
and  better  aid  mounted  Soldiers  in  field  settings. 

The  ATR  preference  was  also  significantly  correlated  with  Attentional  Control  Scores,  thus 
indicating  a  relationship  between  preference  for  the  tactile  ATR  display  and  higher  attentional 
control.  Since  the  dual  visual  task  of  operating  the  TCU  and  the  gunner  station  places  a  large 
burden  on  the  visual  resources,  the  tactile  ATR  display  may  allow  limited  attention  resources  to 
be  time  shared  in  a  more  effective  manner  by  transferring  a  large  amount  of  the  search  task  in  the 
gunnery  station  from  the  visual  to  the  tactile  domain  with  the  use  of  the  ATR,  particularly  for 
those  with  higher  reported  attentional  control.  However,  a  cautionary  note  is  offered  since  the 
ratings  data  may  not  be  truly  interval  in  nature  and  therefore,  the  correlations  may  not  provide 
entirely  reliable  information.  Additional  research  should  be  directed  at  using  multimodal  displays 
to  overcome  spatial  ability  and  attentional  limitations  in  these  demanding  multi-task  environments. 


5  Conclusions 


In  this  study,  we  conducted  a  simulation  experiment  and  examined  the  effectiveness  of  ATR 
capabilities  (delivered  through  tactile  cueing  or  a  combination  of  tactile  and  visual  cueing)  for 
enhancing  the  performance  of  gunners  who  also  had  to  simultaneously  operate  a  UGV  and 
maintain  effective  communication  with  fellow  crew  members.  We  did  not  include  a  visual- 
cueing-only  condition  and  decided  to  focus  on  the  tactile  display  so  we  could  determine  the 
value  of  providing  non- visually  dependent  aid  to  the  operators  to  help  them  with  their  tasks  in 
the  visually  intensive  environment  we  created.  Since  both  types  of  cues  were  provided  in  the 
TacVis  condition,  participants  could  choose  to  use  either  type  or  both.  Therefore,  we  could  still 
examine  the  effect  of  using  visual  cueing  predominantly,  although  admittedly,  not  in  a  statistically 
meaningful  way,  since  only  15%  of  our  participants  relied  primarily  on  visual  cueing. 
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Overall,  our  findings  suggest  that,  in  a  multi-tasking  environment  such  as  the  one  simulated  in  this 
study,  automation  (i.e.,  ATR  in  this  study)  for  one  task  benefits  not  only  the  automated  task  but 
also  the  concurrent  tasks  (i.e.,  robotics  and  communication  in  this  case).  However,  operators  may 
develop  over-reliance  on  the  ATR  for  their  tasks  and  may  overlook  other  developments  that  are  not 
detected  by  the  system  (e.g.,  the  neutral  targets  in  the  current  study).  Since  the  reliability  level  of 
the  ATR  was  set  at  100%  in  this  study,  it  would  be  interesting  to  see  how  ATR  with  imperfect 
reliability  would  affect  an  operator’s  visual  attention.  In  an  ensuing  study,  both  false-alarm-prone 
and  miss-prone  ATR  will  be  simulated  and  their  effects  will  be  examined.  Additionally,  we  select 
personnel  for  simultaneously  performing  gunnery  and  robotic  tasks,  it  might  be  beneficial  to  take 
into  account  their  spatial  ability.  Chen  et  al.  (in  press)  and  Chen  and  Joyner  (2006)  and  the  current 
study  all  demonstrated  the  superior  performance  by  those  with  higher  spatial  ability.  It  is 
especially  important  if  ATR  is  not  available  to  assist  the  operators  with  their  tasks.  These  data  on 
individual  differences  can  be  used  in  future  IMPRINT  modeling  efforts  as  input  data  to  modeling 
tasks  and  therefore  enhance  future  model  analyses.  Finally,  both  visual  and  tactile  cueing  should 
be  provided  in  future  military  mounted  environments  such  as  the  one  simulated  in  the  current 
study.  It  seems  that  low-SpA  individuals  prefer  visual  cueing  over  tactile  cueing,  although  tactile 
display  would  be  more  effective  in  a  highly  visual  environment  (so  visual  attention  can  be  devoted 
to  the  tasks,  not  on  the  cues).  It  is  likely  that  training  interventions  can  be  devised  to  help  these 
low-SpA  individuals  better  employ  tactile  information. 
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Appendix  A.  Demographic  Questionnaire 


Participant# _  Age _  Major _  Date _  Gender _ 

1 .  What  is  the  highest  level  of  education  you  have  had? 

Less  than  4  yrs  of  college _  Completed  4  yrs  of  college _  Other _ 

2.  When  did  you  use  computers  in  your  education?  (Circle  all  that  apply) 

Grade  School  Jr.  High  High  School 

Technical  School  College  Did  Not  Use 

3.  Where  do  you  currently  use  a  computer?  (Circle  all  that  apply) 

Home  Work  Library  Other _  Do  Not  Use 

4.  For  each  of  the  following  questions,  circle  the  response  that  best  describes  you. 

How  often  do  you: 

Use  a  mouse?  Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

Use  a  joystick?  Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

Use  a  touch  screen?  Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

Use  icon-based  programs/software? 

Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 
Use  programs/software  with  pull-down  menus? 

Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 
Use  graphics/drawing  features  in  software  packages? 

Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 
Use  E-mail?  Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

Operate  a  radio  controlled  vehicle  (car,  boat,  or  plane)? 

Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

Play  computer/video  games? 

Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

5.  Which  type(s)  of  computer/video  games  do  you  most  often  play  if  you  play  at  least  once  every  few  months? 

6.  Which  of  the  following  best  describes  your  expertise  with  computer?  (check  V  one) 

_ Novice 

_ Good  with  one  type  of  software  package  (such  as  word  processing  or  slides) 

_ Good  with  several  software  packages 

_ Can  program  in  one  language  and  use  several  software  packages 

_ Can  program  in  several  languages  and  use  several  software  packages 

7.  Are  you  in  your  usual  state  of  health  physically?  YES  NO 
If  NO,  please  briefly  explain: 

8.  How  many  hours  of  sleep  did  you  get  last  night? _ hours 

9.  Do  you  have  normal  color  vision?  YES  NO 

10.  Do  you  have  prior  military  service?  YES  NO  If  Yes,  how  long _ 
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Intentionally  left  blank 
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Appendix  B.  Attentional  Control  Survey 


For  each  of  the  following  questions,  circle  the  response  that  best  describes  you. 

It  is  very  hard  for  me  to  concentrate  on  a  difficult  task  when  there  are  noises  around. 

Almost  never,  Sometimes,  Often,  Always 

When  I  need  to  concentrate  and  solve  a  problem,  I  have  trouble  focusing  my  attention. 

Almost  never,  Sometimes,  Often,  Always 

When  I  am  working  hard  on  something,  I  still  get  distracted  by  events  around  me. 

Almost  never,  Sometimes,  Often,  Always 

My  concentration  is  good  even  if  there  is  music  in  the  room  around  me. 

Almost  never,  Sometimes,  Often,  Always 

When  concentrating,  I  can  focus  my  attention  so  that  I  become  unaware  of  what’s  going  on  in  the  room  around 
me.  Almost  never,  Sometimes,  Often,  Always 

When  I  am  reading  or  studying,  I  am  easily  distracted  if  there  are  people  talking  in  the  same  room. 

Almost  never,  Sometimes,  Often,  Always 

When  trying  to  focus  my  attention  on  something,  I  have  difficulty  blocking  out  distracting  thoughts. 

Almost  never,  Sometimes,  Often,  Always 

I  have  a  hard  time  concentrating  when  I’m  excited  about  something. 

Almost  never,  Sometimes,  Often,  Always 

When  concentrating,  I  ignore  feelings  of  hunger  or  thirst.  Almost  never,  Sometimes,  Often,  Always 


I  can  quickly  switch  from  one  task  to  another.  Almost  never,  Sometimes,  Often,  Always 

It  takes  me  a  while  to  get  really  involved  in  a  new  task.  Almost  never,  Sometimes,  Often,  Always 

It  is  difficult  for  me  to  coordinate  my  attention  between  the  listening  and  writing  required  when  taking  notes 
during  lectures.  Almost  never,  Sometimes,  Often,  Always 

I  can  become  interested  in  a  new  topic  very  quickly  when  I  need  to. 

Almost  never,  Sometimes,  Often,  Always 

It  is  easy  for  me  to  read  or  write  while  I’m  also  talking  on  the  phone. 

Almost  never,  Sometimes,  Often,  Always 

I  have  trouble  carrying  on  two  conversations  at  once.  Almost  never,  Sometimes,  Often,  Always 

I  have  a  hard  time  coming  up  with  new  ideas  quickly.  Almost  never,  Sometimes,  Often,  Always 

After  being  interrupted  or  distracted,  I  can  easily  shift  my  attention  back  to  what  I  was  doing  before. 

Almost  never,  Sometimes,  Often,  Always 

When  a  distracting  thought  comes  to  mind,  it  is  easy  for  me  to  shift  my  attention  away  from  it. 

Almost  never,  Sometimes,  Often,  Always 

It  is  easy  for  me  to  alternate  between  two  different  tasks.  Almost  never,  Sometimes,  Often,  Always 
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It  is  hard  for  me  to  break  from  one  way  of  thinking  about  something  and  look  at  it  from  another  point  of  view. 

Almost  never,  Sometimes,  Often,  Always 
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Appendix  C.  NASA-TLX  Questionnaire 


Please  rate  your  overall  impression  of  demands  imposed  on  you  during  the  exercise. 

1 .  Mental  Demand:  How  much  mental  and  perceptual  activity  was  required  (e.g.,  thinking,  looking, 
searching,  etc.)?  Was  the  task  easy  or  demanding,  simple  or  complex,  exacting  or  forgiving? 

LOW  I— I— I— I— I— I— I— I— I— |  HIGH 
123456789  10 

2.  Physical  Demand:  How  much  physical  activity  was  required  (e.g.,  pushing,  pulling,  turning, 
controlling,  activating,  etc.)?  Was  the  task  easy  or  demanding,  slow  or  brisk,  slack  or  strenuous,  restful  or 
laborious? 


LOW  I— I— I— I— I— I— I— I— I— |  HIGH 
123456789  10 

3.  Temporal  Demand:  How  much  time  pressure  did  you  feel  due  to  the  rate  or  pace  at  which  the  task  or 
task  elements  occurred?  Was  the  pace  slow  and  leisurely  or  rapid  and  frantic? 

LOW  I— I— I— I— I— I— I— I— I— |  HIGH 
123456789  10 

4.  Level  of  Effort:  How  hard  did  you  have  to  work  (mentally  and  physically)  to  accomplish  your  level  of 
performance? 


LOW  l-l-l-H-l-l-l-l-l  HIGH 
123456789  10 

5.  Level  of  Frustration:  How  insecure,  discouraged,  irritated,  stressed  and  annoyed  versus  secure, 
gratified,  content,  relaxed  and  complacent  did  you  feel  during  the  task? 

LOW  I— |— I— |— |— |— |— I— I— |  HIGH 
123456789  10 

6.  Performance:  How  successful  do  you  think  you  were  in  accomplishing  the  goals  of  the  task  set  by  the 
experimenter  (or  yourself)?  How  satisfied  were  you  with  your  performance  in  accomplishing  these  goals? 

LOW  I— |— H— I— I— I— h-i— I  HIGH 
123456789  10 
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Appendix  D.  Simulator  Sickness  (Current  Health  Status)  Questionnaire 


ID _  Time  &  Date _ 

Instructions:  Please  indicate  how  you  feel  right  now  in  the  following  areas,  by  circling  the  word  that 

applies. 


1. 

General  Discomfort 

None 

Slight 

Moderate 

Severe 

2. 

Fatigue 

None 

Slight 

Moderate 

Severe 

3. 

Headache 

None 

Slight 

Moderate 

Severe 

4. 

Eye  Strain 

None 

Slight 

Moderate 

Severe 

5. 

Difficulty  Focusing 

None 

Slight 

Moderate 

Severe 

6. 

Increased  Salivation 

None 

Slight 

Moderate 

Severe 

7. 

Sweating 

None 

Slight 

Moderate 

Severe 

8. 

Nausea 

None 

Slight 

Moderate 

Severe 

9. 

Difficulty  Concentrating 

None 

Slight 

Moderate 

Severe 

10. 

Fullness  of  Head 

None 

Slight 

Moderate 

Severe 

11. 

Blurred  vision 

None 

Slight 

Moderate 

Severe 

12. 

Dizzy  (Eyes  Open) 

None 

Slight 

Moderate 

Severe 

13. 

Dizzy  (Eyes  Closed) 

None 

Slight 

Moderate 

Severe 

14. 

Vertigo* 

None 

Slight 

Moderate 

Severe 

15. 

Stomach  Awareness** 

None 

Slight 

Moderate 

Severe 

16. 

Burping 

None 

Slight 

Moderate 

Severe 

Vertigo  is  a  disordered  state  in  which  the  person  or  his/her  surroundings  seem  to  whirl  dizzily:  giddiness 
**  Stomach  awareness  is  usually  used  to  indicate  a  feeling  of  discomfort  which  is  just  short  of  nausea. 


Are  there  any  other  symptoms  you  are  experiencing  right  now?  If  so,  please  describe  the  symptom(s)  and 
rate  its/their  severity  below.  Use  the  other  side  if  necessary. 
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Intentionally  left  blank 
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Appendix  E.  Usability  Questionnaire 


1.  The  gunner  station  should  only  have  the  visual  ATR  display. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

2.  I  made  use  of  both  the  visual  and  tactile  ATR  displays. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

3.  I  sometimes  felt  ‘lost’  using  the  tactile  ATR  display. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

4.  I  sometimes  felt  ‘lost’  using  the  visual+tactile  display. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

5.  The  tactile  ATR  display  was  intuitive  and  made  it  easy  to 
determine  the  direction  of  targets. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

6.  The  visual+tactile  ATR  display  was  helpful  when  I  had  to 
teleoperate  the  UGV. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

7.  The  visual+tactile  ATR  display  was  helpful  when  the  UGV  was 
semi-autonomous. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

8.  The  gunner  station  should  not  have  an  ATR  display. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

9.  The  tactile  ATR  display  was  confusing. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

10.  The  visual+tactile  ATR  display  was  intuitive  and  made  it  easy  to 
determine  the  direction  of  targets. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 


Comments 
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11.  The  visual+tactile  ATR  display  was  confusing. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

12.  The  tactile  ATR  display  was  annoying. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

13.  The  visual+tactile  ATR  display  was  annoying. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

14.  The  tactile  ATR  display  improved  my  performance  on  the  gunner 
task. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 

15.  The  visual+tactile  ATR  display  improved  my  performance  on  the 
gunner  task. 

Strongly  DISAGREE  | — | — | — | — | — | — |  Strongly  AGREE  N/A 

1  2  3  4  5  6  7 


Which  of  the  following  best  describes  your  source  of  ATR  information  when  you  had  access  to 
both  the  visual  and  the  tactile  displays  (please  circle  ONE  answer  only): 

1 .  entirely  visual 

2.  predominately  visual 

3.  both  visual  and  tactile 

4.  predominately  tactile 

5.  entirely  tactile 
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Appendix  F.  Scoring  Procedure  for  the  Simulator  Sickness  Questionnaire 


Symptoms  scored  0  (None)  -  3  (Severe) 

Nausea  (Raw)  -  Sum  of  General  discomfort,  increased  salivation,  sweating,  nausea,  diff 
concentrating,  stomach  awareness,  burping 

Nausea  sub  scale  =  Nausea  (Raw)  x  9.54 

Oculomotor  -  Sum  of  general  discomfort,  fatigue,  headache,  eye  strain,  diff  focusing,  diff 
concentrating,  blurred  vision 

Oculomotor  sub  scale  =  Nausea  (Raw)  x  7.58 

Disorientation  -  Sum  of  diff  focusing,  nausea,  fullness  of  head,  blurred  vision,  dizzy 
(eyes  open),  dizzy  (eyes  closed),  vertigo 

Disorientation  sub  scale  =  Nausea  (Raw)  x  13.92 

TSS  =  [Nausea  (Raw)  +  Oculomotor  (Raw)  +  Disorientation  (Raw)]  x  3.74 
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Glossary  of  Acronyms 


ANOVA 

analysis  of  variance 

ARL 

Army  Research  Laboratory 

ATR 

aided  target  recognition 

FCS 

Future  Combat  System 

FOV 

field  of  view 

IMPRINT 

Improved  Performance  Research  Integration  Tool 

LOS 

line  of  sight 

LSD 

Least  Significant  Difference 

MCS 

mounted  combat  system 

NASA-TLX 

National  Aeronautics  and  Space  Administration-task  load  index 

OTW 

out  the  window 

PAC 

perceived  attentional  control 

RCTA 

Robotics  Collaborative  Technology  Alliance 

RSTA 

reconnaissance,  surveillance,  and  target  acquisition 

SIL 

system  integration  laboratory 

SpA 

spatial  ability 

SSQ 

Simulator  Sickness  Questionnaire 

TCU 

tactical  control  unit 

TS 

total  severity 

TSS 

Total  Severity  Score 

UCF 

University  of  Central  Florida 

UGV 

unmanned  ground  vehicle 
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NO.  OF  NO.  OF 

COPIES  ORGANIZATION  COPIES  ORGANIZATION 


1  DEFENSE  TECHNICAL 
(PDF  INFORMATION  CTR 
ONLY)  DTICOCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BEL  VOIR  VA  22060-6218 

1  US  ARMY  RSRCH  DEV  &  ENGRG  CMD 
SYSTEMS  OF  SYSTEMS 
INTEGRATION 
AMSRD  SS  T 
6000  6TH  ST  STE  100 
FORT  BEL  VOIR  VA  22060-5608 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMNE  ALC  IMS 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1 197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  Cl  OK  TL 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1 197 

2  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  CS  OK  T 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1 197 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  ML  J  MARTIN 
MYER  CENTER  RM  2D3 1 1 
FT  MONMOUTH  NJ  07703-5601 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MZ  A  DAVISON 
199  E  4TH  ST  STE  C  TECH  PARK  BLDG  2 
FT  LEONARD  WOOD  MO  65473-1949 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MD  T  COOK 
BLDG  5400  RM  C242 
REDSTONE  ARSENAL  AL  35898-7290 

1  COMMANDANT  USAADASCH 

ATTN  AMSRD  ARL  HR  ME  J  HAWLEY 
5800  CARTER  RD 
FT  BLISS  TX  79916-3802 


1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MM  DR  V  RICE-BERG 

BLDG  4011  RM  217 

1750  GREELEY  RD 

FT  SAM  HOUSTON  TX  78234-5002 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MG  R  SPINE 
BUILDING  333 

PICATINNY  ARSENAL  NJ  07806-5000 

1  ARL  HRED  ARMC  FLD  ELMT 

ATTN  AMSRD  ARL  HR  MH  C  BURNS 
BLDG  1467B  ROOM  336 
THIRD  AVENUE 
FT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 
AWC  FIELD  ELEMENT 
ATTN  AMSRD  ARL  HR  MJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MK  MR  J  REINHART 
10125  KINGMAN  RD 
FT  BEL  VOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MV  HQ  USAOTC 
S  MIDDLEBROOKS 
91012  STATION  AVE  ROOM  348 
FT  HOOD  TX  76544-5073 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MY  M  BARNES 
2520  HEALY  AVE  STE  1172  BLDG  51005 
FT  HUACHUCA  AZ  85613-7069 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MP  D  UNGV ARSKY 
BATTLE  CMD  BATTLE  LAB 
415  SHERMAN  AVE  UNIT  3 
FT  LEAVENWORTH  KS  66027-2326 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MJF  J  HANSBERGER 
JFCOM  JOINT  EXPERIMENTATION  J9 
JOINT  FUTURES  LAB 
115  LAKEVIEW  PARKWAY  SUITE  B 
SUFFOLK  VA  23435 
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NO.  OF 

COPIES  ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MQ  M  R  FLETCHER 
US  ARMY  SBCCOM  NATICK  SOLDIER  CTR 
AMSRD  NSC  WS  E  BLDG  3  RM  343 
NATICK  MA  01760-5020 

2  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MT  J  CHEN 

C  KORTENHAUS 
12350  RESEARCH  PARKWAY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MS  MR  C  MANASCO 
SIGNAL  TOWERS  ROOM  303 
FORT  GORDON  GA  30905-5233 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MU  M  SINGAPORE 
6501  E  1 1  MILE  RD  MAIL  STOP  284 
BLDG  200 A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MF  MR  C  HERNANDEZ 
BLDG  3040  RM  220 
FORT  SILL  OK  73503-5600 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MW  E  REDDEN 
BLDG  4  ROOM  332 
FT  BENNING  GA  31905-5400 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FORT  BRAGG  NC  28310-5000 

1  ARMY  G1 

ATTN  DAPE  MR  B  KNAPP 

300  ARMY  PENTAGON  ROOM  2C489 

WASHINGTON  DC  20310-0300 

1  ARL-HRED  LIAISON 

PHYSICAL  SCIENCES  LAB 
PO  BOX  30002 

LAS  CRUCES  NM  88003-8002 
1  DIRECTOR 

UNIT  OF  ACTION  MANEUVER  BATTLE  LAB 
ATTN  ATZKUA 
BLDG  1101 

FORT  KNOX  KY  40121 


1  DIR  FOR  PERS  TECHNOLOGIES 
DPY  CHIEF  OF  STAFF  PERS 
300  ARMY  PENTAGON  2C733 
WASHINGTON  DC  20310-0300 

1  CODE  1142PS 

OFC  OF  NAVAL  RSCH 
800  N  QUINCY  STREET 
ARLINGTON  VA  22217-5000 

1  CDR 

USA  AEROMEDICAL  RSCH  LAB 

ATTN  LIBRARY 

FORT  RUCKER  AL  36362-5292 

1  US  ARMY  NATICK  RD&E  CTR 
ATTN  STRNCYBA 
NATICK  MA  01760-5020 

1  PEO  STRI 

12350  RSCH  PARKWAY 
ORLANDO  FL  32826-3276 

1  GOVT  PUBLICATIONS  LIB 
409  WILSON  M 

UNIVERSITY  OF  MINNESOTA 
MINNEAPOLIS  MN  55455 

1  HUMAN  FACTORS  ENG  PROGRAM 
DEPT  OF  BIOMEDICAL  ENGNG 
COLLEGE  OF  ENG  &  COMPUTER  SCIENCE 
WRIGHT  STATE  UNIVERSITY 
DAYTON  OH  45435 

1  DIR  AMC-FIELD  ASSIST  IN 
SCIENCE  &  TECHNOLOGY 
ATTN  AMC-FAST 
FT  BEL  VOIR  VA  22060-5606 

ABERDEEN  PROVING  GROUND 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  Cl  OK  (TECH  LIB) 
BLDG  4600 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  Cl  OK  S  FOPPIANO 
BLDG  459 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  HR  MR  F  PARAGALLO 

BLDG  459 
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